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Abstract — Expectation Propagation is a generalization to Belief 
Propagation (BP) in two ways. First, it can be used with any 
exponential family distribution over the cliques in the graph. 
Second, it can impose additional constraints on the marginal 
distributions. We use this second property to impose pair-wise 
marginal distribution constraints in some check nodes of the 
LDPC Tanner graph. These additional constraints allow decoding 
the received codeword when the BP decoder gets stuck. In 
this paper, we first present the new decoding algorithm, whose 
complexity is identical to the BP decoder, and we then prove that 
it is able to decode codewords with a larger fraction of erasures, 
as the block size tends to infinity. The proposed algorithm can be 
also understood as a simplification of the Maxwell decoder, but 
without its computational complexity. We also illustrate that the 
new algorithm outperforms the BP decoder for finite block-size 
codes. 

I. Introduction 

In this paper we propose a new algorithm to decode low- 
density parity-check (LDPC) codes over the binary erasure 
channel (BEC). The proposed algorithm has a better perfor- 
mance than the traditional Belief Propagation (BP) decoder 
with the same computational complexity. The analysis of the 
BP decoder over independent and identically distributed BEC 
with erasure probability e has been detailed in (TJ, 0, 0, in 
which its limiting performance and optimization are addressed. 
Once we represent the LDPC code by the bipartite graph 
induced by its parity-check matrix (Tanner Graph [4|), the 
BP decoding algorithm can be easily described. The degree of 
a check node is the number of variable nodes connected to it 
in the bipartite graph, and vice-versa. 

We initialize the decoder by removing from the graph all 
the variable nodes corresponding to non-erased bits. We also 
remove all the connections from these variable nodes. After 
removing a variable node whose value was one, we change 
the parity of the check node(s) it was connected to. After the 
initialization stage, the BP algorithm proceeds by removing a 
check node and a variable node in each step: 

1) It looks for any check node linked to a single variable 
node (a check node of degree one). The BP decoder 
copies the parity of this check node into the variable node 
and removes the check node. 



2) Second, it removes the variable node that we have just 
de-erased. If the variable was a one, it changes the parity 
of the check node(s) it was connected to. 

3) It repeats Steps 1 and 2 until all the variable nodes have 
been removed, successfully finishing the decoding of the 
received word, or until there are no degree-one check 
nodes left, yielding an unsuccessful decoding. 

The asymptotic analysis of the BP performance decod- 
ing LDPC codes over the BEC is derived in (TJ, J5J and 
capacity-achieving degree distributions are presented in (JJ, 
0. However, for graphs with cycles, the performance of the 
BP decoder for any finite-length code is always inferior to the 
maximum a posteriori (MAP) decoder. For example, in [6| it 
is shown that a BP decoder for a rate- 1/2 regular LDPC code 
with three ones per column and six ones per row can decode 
channels with erasure probability up to e = 0.4294, which is 
typically denoted as the BP capacity, as the block size tends to 
infinity, while the limiting MAP performance reaches 0.48815, 
which is denoted as the MAP capacity. 

In [6], the authors proposed the Maxwell decoder to achieve 
the MAP solution for decoding LDPC codes over BEC. Once 
the BP gets stuck, because there are no additional degree-one 
check nodes, the Maxwell decoder assumes that the bits of 
a set of variable nodes are known so there is one degree-one 
check node that allows restarting the BP algorithm. This step is 
repeated as many times as needed, until all variable nodes have 
been accounted for. The complexity of the Maxwell decoder 
grows exponentially with the number of guessed variables and 
there is no meaningful upper bound on how many we need to 
assume known. Hence, it is an impractical algorithm to decode 
LDPC codes over BEC, although it is a useful tool to examine 
the code performance and derive the MAP capacity [6|. The 
idea of a BP decoder with variable guessing was first proposed 
in G) for short-length LDPC codes, where the complexity of 
the algorithm is relatively small. 

In this paper, we propose a new algorithm that is able to 
continue decoding LDPC codes for BEC after an unsuccessful 
BP decoding. This algorithm improves the performance of the 
BP decoder for finite length codes at a similar complexity. We 
include the analysis of the asymptotic case to show that the 
achieved capacity is higher than the BP one. Our decoding 



algorithm borrows from the Tree-structured approximations 
for Expectation Propagation [8| and we refer to it as the 
TEP algorithm. The algorithm in (8) can be understood as 
a generalization of the BP decoder, in the sense that the 
variable nodes are linked to impose some pair-wise marginal 
constraints, instead of trying to compute only independent 
marginal distributions. 

The rest of the paper is organized as follows. Section |II] is 
devoted to introducing the TEP algorithm for decoding LDPC 
codes. In Section III we analyze a simplified version of the 
TEP decoder to prove that it achieves a higher capacity than 
BP. In Section |IV| we compare the performance of the TEP 
and BP decoders with finite-length regular LDPC codes. We 
conclude in Section VI with some final comments. 
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Fig. 1. In (a) we show two variable nodes, V\ and V2, that share a check 
node of degree two. In (b), we show the graph once Pi and V2 have been 
removed. If Pi is parity one, the parities of P2 and P3 are reversed. 



II. TEP ALGORITHM 

The TEP decoder starts once the BP gets stuck. In a way, it 
works as the Maxwell decoder in which some variable nodes 
are assumed to be known to continue decoding. But the TEP 
complexity is identical to the BP, as it removes a check node 
and a variable node in each step and its complexity does not 
depend on the number of assumed variable nodes. 

The Tanner graph of the LDPC code, denoted as T, has 
n variable nodes Vi,...,V n and n(l — R) check nodes 
Pi, ... , P n (i-R), where r is the rate of the code. The degree 
of a variable node V m and a check node Pj is, respectively, 
denoted by dy m and dp.. The graph T is reduced by removing 
the non-erasured variables and performing the BP decoder 
until there are no degree-one check nodes left. We refer to the 
graph at the end of the BP as Tbp- The TEP decoder works 
over Tbp using the degree-two check nodes. A check node of 
degree two tells us that the variable nodes connected to it are 
either equal, if the check has parity zero, or different otherwise. 
Our algorithm chooses any check node with degree two and 
removes it from the graph together with one of the variable 
nodes connected to it and the two associated edges. Then it 
reconnects to the remaining variable node all the check nodes 
that were connected to the removed variable node. Finally, 
the parities of the check nodes connected to the remaining 
variable node have to be reversed if the removed degree-two 
check node had parity one. We sketch the procedure in Fig. [TJ 
where the algorithm selects the check node P\ and removes it 
along with the variable V^- We can observe in Fig. [TJb) that 
Vi is now connected to the check nodes P2 and P3 and its 
degree is now dv 1 + dy 2 — 2. 

The algorithm removes a check and a variable node per 
iteration, as the BP does. The removal of a check node and 
a variable does not increase the complexity of the decoder, 
as it happens in the Maxwell decoder f6]. Unlike the BP, the 
value of the removed variable node is unknown. This variable 
becomes known, when the remaining variable node in the 
graph has been decoded. Hence, the decoder only has to store 
a list of the removed variables and the associated remaining 
variables in the graph. In the example of Fig. [TJ it means that 
V2 would be known once V\ has been de-erased. 



This procedure continues removing check and variable 
nodes, but it does not set any of the variables. To be able 
to set any variable node, we need to find a check node with 
degree two such that the two variable nodes connected to it 
also share another check node with degree three, as illustrated 
in Fig. [2ja). When we remove the check node P 3 and the 
variable node V%, the check node P4 is now degree one, as 
illustrated in Fig. |2|b), and we can restart the BP algorithm. 




(a) (b) 

Fig. 2. In (a), the variables Vi and V2 are connected to a two-degree check 
node, P3, and they also share a check node of degree three, P4. In (b) we 
show the graph once the algorithm has removed P3 and V2. 

When the TEP algorithm starts running over the graph 
Tbp, it is very unlikely that the two variable nodes in a 
check node of degree two also share a check node of degree 
three. However, as we remove variable and check nodes the 
probability of this happening grows, since we are reducing 
the number of nodes in the graph and increasing the degree 
of the remaining variable nodes. In the following section, we 
prove that the growth of this probability is exponential with 
the number of iterations of the TEP decoder. In each iteration 
we remove a check node of degree two and a variable node 
connected to it. Eventually, when a check node of degree one 
is created, we can restart the BP algorithm and correct error 
patterns undecodable by the BP decoder. The TEP decoder 



halts either when the graph runs out of check nodes of degree 
one and two or when all the variables have been de-erased. 

III. Analysis of the TEP decoder 

We now analyze a simplification of the above procedure 
and show that it achieves a higher capacity than BP. The TEP 
decoder first removes all the check nodes of degree two, as 
explained in Section |IIJ and then it runs the BP decoder. This 
modification allows applying the results in [ 1 1, in which the BP 
capacity for BEC was computed, to analyze the TEP decoder. 
We consider a LDPC ensemble LDPC[X(x), p(x), n] defined 
by the code length n and the edge degree distribution (DD) 
pair [X(x),p(x)} 0: 
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Fig. 3. Given the check node Pj, in (a) the variable nodes V v and V q 
share only the check node Pj, In (b), they share another one of degree m £ 
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p( x ) = z^ 4-1 ' 



(i) 
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where A; represents the fraction of edges with left degree i in 
the graph and p. L is the fraction of edges with right degree i. 
The left (right) degree of an edge is the degree of the variable 
(check) node it is connected to. The total number of edges in 
the graph is denoted by E and it can be readily check that 



E 



(3) 



The passage of time during the decoding process is scaled so 
that each time unit of length At corresponds to one iteration 
of the decoding process as in (TJ. In one iteration, the TEP 
removes a check node of degree two and one variable. 

Let define k(t) and such that Li(i) — k(t)E and 

R%{t) — h(t)E are, respectively, the number of edges with 
left and right degree i in the graph at time t. We denote e(t) 
as the number of edges in the graph at time t divided by 
E. Hence, li(t)/e(t) for i = 1, . . . ,l max and ri(t)/e(t) for 
i = l,..., r max are the coefficients of the DD pair that defines 
the graph at time t. In the analysis of the BP decoder in (TJ, 
the evolution of the DD pair in the limiting case, as n tends to 
infinity, is described by a set of differential equations. The BP 
decoder has analytical solution, which holds while r% (t) > 0. 
In particular, the solution for r\ (t) is given by: 



ri (x) = e\{x) [x - 1 + p(l - eX(x))] . 



(4) 



The BP capacity, epp, is the maximum probability of erasure 
such that ri(x) > Vx e (0, 1]. 

A. TEP analysis 

For a BEC with e > epp, the solution of r\{x) in Q 
touches for x = Xbp > and the BP decoder gets stuck at 
this point. The residual graph is Tbp and it is defined by the 
DD pair 
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where npp is the number of remaining variables. The TEP 
decoder takes the graph Tbp as input and removes one check 
node of degree two and one variable per iteration. While in the 
BP decoder, the basic step has an unique formulation, now we 
have two different situations. Once a check node Pj of degree 
dp. =2 has been found: 

A. The variable nodes connected with the check, V p and V q , 
do not share another check node. See Fig. [3fa). 

B. V p and V q share another check node Pi of degree dp l = m, 
where m e {2, ... , r max }. See Fig. |5fb). 

In the ensemble in |5]l and provided that V p and V q are 
connected to Pj of degree two, pb(xbp) represents the 
probability that the variable nodes also share another check 
node. This probability can be approximated as follows: 
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e(x B p)E 



(6) 



where a(xsp) = J2i^i( x Bp) /e(xpp) is the average edge 
left degree for the ensemble in (|5j. This expression is proven 
in ifTUl . In the limiting case, as E —> oo, then pb(xbp) —> 0. 

We have divided the DD evolution during the decoding pro- 
cess in two stages, each one described by a set of differential 
equations: 

• Stage A. The equations describe the DD evolution assum- 
ing that, in one iteration of the TEP decoder, the variable 
nodes connected to any check node of degree two do not 
share another check node. 

• Stage B. The equations describe the DD evolution when, 
in one iteration of the TEP decoder, the variable nodes 
connected to a check node of degree two always share 
another check node. 

The Stage A equations model the evolution of the DD pair 
at the beginning of the TEP decoding. The solution of the 
equations shows the probability pp(i) grows exponentially 
with time. Although the Stage A equations assume p B (t) zero 
or negligible, we use them to approximate the evolution of 
the DD pair until pb(£) = 1. Notice that this is a conservative 
solution in the sense we are ignoring the check nodes of degree 
one that could be created before ps{t) is one. The Stage A 
lasts between t = 0, when the TEP starts, and t = t^. We 



define as the time in which pg (t) is either 1 or the graph 
runs out of check nodes of degree two. In the first case, Stage 
B starts. In the second one, the TEP decoder finishes without 
creating check nodes of degree one and it cannot improve the 
BP solution. 

In the Stage B, ps(£) = 1 and the variables connected to 
a check node of degree two also share another check node. 
Hence, check nodes of degree one can be created. The Stage 
B equations hold until the decoder removes all the check nodes 
of degree two. At this time, denoted as ts, the BP decoder is 
run again. 

B. Stage A of decoding 

Suppose that, at time t, we remove the check node Pj in Fig. 
[3^a), which is connected to V p and V q . lfV p is the remaining 
variable, its degree is now dv + dy — 2. From the edge 
perspective, the graph losses dy p edges with left degree dy v 
and dy q edges with left degree dy q and gains dy p + dy q — 2 
edges with left degree dy p + dy q — 2. The probability that dy p 
or d Vq are i is k(t)/e(t) for i = 1, . . . ,l max (t). 

To derive the differential equation that describes the evolu- 
tion of li(t) for i — 1, . . . , l m ax(t), we compute the expected 
variation of the L. t (t) distribution, E [L t (t + At) - L,-(t)]. 
Hence, we are interested in cases where we have Li(t+ At) ^ 
Li(t). For i > 3: 

1) U(t + At) = Li(t) - i if (d Vp = i XOR d Vq = i) AND 
dv p + d Vq - 2 ^ i. 

2) U{t + At) = Li{t) - 2i if d Vp = d Vq = i. 

3) Li(t + At) = Li\t) + i if d Vp f i, d Vq ^ i AND d Vp + 
d Vq -2 = i. 

The probability of each case is respectively, 

,*i(t) ( k{t) hit) 
J e(t) \ e(i) e(t) 

'kit)" 1 
At) 

lq(t) k+2-q(t) 
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(7) 



(8) 



P3,i = 
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e(t) e(t) 



(9) 



For i = 2, L2(t + At) ^ L2(t) only in Cases 1 and 3. The 
expected variation of the Li(t + At) distribution with respect 

to Li{t) is 

E [L 2 (t + At) - L 2 (t)} - -2(pi l2 - P3,a), (10) 
E[L i {t + At)-L i {t)] = -i( Plii + 2p 2ti -p 3ti ). (11) 



If we set At = E 1 and n 
be rewritten as (TJ: 



then ( fT0) > an d ( fTTj l can 



dhjt) 

dt 
dkjt) 

dt 



= -2 (pi,2 - P3,2) , i 
= -i (pi ti + 2p 2 .i ~P3,i) 



(12) 
i>3. (13) 



grows as 0(2'). In each iteration, two edges of right degree 
two are lost. Therefore, E [R 2 (t + At) - R 2 (t)] = -2 and 
the differential equation for r 2 (t) is 

dr 2 (t) 



dt 



r 2 (t) = r 2 (0)-2t = r 2 (x BP )-2t. (14) 



Equations 12 and [13] represent a system of coupled nonlinear 
differential equations whose solution has to be computed by 
numerical methods. The solution shows that the average left 
degree a(t) = ^2nili{t) /e(t) grows exponentially with time 
and so does in ([§}. 

Stage A can either finish, because it runs out of degree 
two check nodes or because all the variables nodes that share 
a check node of degree two also share another check node, 
which means that psit) = 1. If the end time of Stage A, 
denoted as t^, is t max ,A = t 2 (xbp)/2, then r 2 (t) in fHj ) 
is zero and we cannot create new degree one-check nodes. 



Otherwise, if tA < t T< 



we enter Stage B, in which 



the decoder creates new degree-one check nodes and we can 
restart the BP decoder. 

C. Stage B 

The graph at the beginning of this stage is defined by the 
solution of ( ]12[ i, ( fT3j ) and ( p~4[ > at t = tA- We are in a situation 
similar to Fig. [3jb) and we select a check node as Pj. The 
variables V p and V q are also linked to Pi with degree dp, = m. 
If dy p and dy are the degree of the variables, the remaining 
variable node has degree dy p + dy q — 4 Q The check node 
Pi losses two edges and its degree reduces to dp, = m — 2. 
The equations describing the evolution of the li (t) distribution 
for i = 2, . . . , Imn-rti) can be derived as in Section III-B 



Regarding to the r m (t) distribution, the graph looses m edges 
of right degree m and gains to — 2 edges of right degree to — 2. 
As the check node Pi has degree dp, = m with probability 
Tm{t)/e{t), the r m (t) evolution can be described as follows 
HI: 



dn(t) r 3 (t) 



dt 
dr 2 (t) 

dt 
dr m (t) 



e(t)' 
>*(*) 



ra(i) 



e(t) e(t) 

n+2{t) 



(15) 



(16) 



.(*) 



dt \ e(t) e(t) 

where the term —2 in ( fT6| ) represents the lost edges of the 
check node Pj. Let ts represents the time when r 2 (t) = 0. At 
t = ts, all the check nodes of degree two have been removed. 
Stage B ends and the BP decoder is begins. The BP input is 
the DD solution of Stage B equations at ts- 

D. TEP decoder solution for a regular LDPC ensemble 

In this section, we estimate the TEP decoder capacity, de- 
noted as £tep, for a regular LDPC ensemble with X(x) 



and p(x) 



The BP capacity is e B p = 0.4294 0], 0- We 



solve ( |12) , ( ff3| l and ([14} using Euler's method and compute 
£max,A = 0.4315, which is the maximum channel probability 



The initial conditions for t = are given by the BP ensemble 
defined in (|5}. The maximum left degree l ma x(t) grows 

according to l max (t + At) = 2l max (t) — 2 and, hence, l max (t) 'The probability that V p and V q share a third check node P u in negligible. 
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Fig. 4. WER performance of the TEP decoder (solid line) and the BP decoder 
(dashed line) for a regular (3,6) LDPC ensemble and code lengths n = 2 l 
with i = 9(o), 10(+), ll(v)> 12 (n). Each curve has been averaged for 100 
different samples of the ensemble. 



of erasure such that the distribution has = 1 before the 

graph runs out of check nodes of degree two. 

For e m ax,A and the solution of Stage A equations at tA, we 
solve the Stage B equations. The Stage B equations hold until 
the graph has no check nodes of degree two, which happens 
at t = ts- The DD pair at t = ts is specially favorable 
for the BP decoder: it has a certain number of check nodes of 
degree one that can be connected to variable nodes of arbitrary 
high degree. When the BP decoder removes one check node 
of degree one and a variable, the number of edges removed 
is very high and also the probability that new check nodes of 
degree one can be created. As expected, the BP solution, given 
the input ensemble defined by the DD at t = ts, verifies the 
condition r\{x) > Vx 6 (0, 1]. Hence, the TEP algorithm 
for n —> oo can decode at least up to e = 0.4315. This value 
is just a lower bound of the capacity of the TEP decoder, since 
the decoder actually creates more check nodes of degree one 
than the ones provided by Stage B solution. In any case, we 
have shown that ctep > 0.4315 > ebp- 



IV. Experimental results for finite-length codes 



V. Conclusions 

In this paper, we have proposed a new decoding algorithm 
for LDPC codes over the BEC. The TEP algorithm exhibits 
a significant improvement compared to the BP decoder with 
identical computational complexity. The TEP algorithm bor- 
rows from the Tree-structured approximations for Expectation 
Propagation and it is able to continue decoding when BP halts,, 
because there are no degree-one check nodes left. We have 
proved that the TEP decoder achieves a higher capacity than 
the BP by deriving the equations for the DD evolution along 
the decoding process. We have illustrated the results for regular 
LDPC codes. The extension of the analysis to irregular LDPC 
ensembles is immediate and it is carried out in ifTUll . 
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In this section, we illustrate the performance of the TEP 
decoder, as proposed in Section [il] to show that it signifi- 
cantly improves the performance of BP decoder for finite- 
length codes. We have used the regular LDPC ensemble 



analyzed in the above section. In Fig. IV we depict the 
worderror rate (WER) for the TEP decoder, solid lines, and 
the BP decoder, dashed lines, for code lengths n = 2 l with 
i = 9(o), 10(-f), ll(v), 12(D). Each curve has been averaged 
for 100 different samples of the regular ensemble. The TEP 
decoder always improves the BP decoder. 



